The present invention relates generally to data communication systems and methods and, more particularly, to data communication systems and methods in which a number of virtual network interfaces efficiently share hardware resources in, for example, Ethernet-based, scalable and tightly coupled systems.
Ethernet's broad use continues to stimulate dramatic increases in performance and decreases in cost for components commonly used in commercial applications. Many of today's commercial applications tolerate the relatively high latency associated with Ethernet-based systems, however emerging commercial applications, such as multithreaded databases and file systems, will likely require reduced latency. Some specialized network solutions provide reduced latency, but are more expensive than Ethernet-based scalable clusters.
One area in which latency performance can be improved is in the network interface controller (NIC). A NIC is a hardware device that supports communication with a network. As context, consider the exemplary system of FIG. 1. Therein a symmetric multiprocessor (SMP) system 10 includes a number of central processor units (CPUs) 12 which share memory unit 14 via coherence fabric 16. Although SMP 10 is shown as having four processor cores, those skilled in the art will appreciate that SMP 10 can have more or fewer CPUs. SMP 10 sends messages to other SMPs 20 under the control of NIC 18 via Ethernet connections and a fabric (switch) 22. The NIC 18 will typically have a processor (not shown) associated therewith, either as an integral part of the NIC or in the form of a helper processor, so that the NIC has sufficient intelligence to interpret various commands. The fabric 21 will route messages to their intended recipients, although occasionally messages will be dropped such that the system illustrated in FIG. 1 needs to support retransmission of dropped messages.
Although there is only one hardware NIC 18 per SMP 10, 20, many different software programs may be running simultaneously on a given SMP and may have messages to transmit across the system via fabric 22. Thus the NIC 18 needs to be implemented as a shared resource. One approach for sharing the NIC 18 is to require that, as part of the message transmission process, the various software programs call a complex operating system driver to coordinate shared access to the NIC 18. However, this shared access mechanism leads to high software overhead as a time consuming operating system call is required for frequently executed communication operations.
Another approach for sharing the NIC 18 employs virtual network interface controllers (VNICs) to provide a distinct interface for each of the multiple programs that share that NIC. A VNIC is a user-level software interface that is used, by a program, to communicate directly with a NIC. A VNIC can be implemented within a special region of a user's memory space where actions, such as the reading and writing of data, are used to direct the NIC to carry out communication operations. A special communication library can be provided to translate higher level communication operations, such as sending a message, into appropriate lower-level actions used to control the NIC.
As shown in FIG. 2, since a number of VNICs 22 operate to share one NIC 18, a priority mechanism 24 is used to determine which VNIC shall receive service from a NIC among a set of competing service requests. However, to further reduce latency, once a VNIC 22 is selected for service by the NIC 18, it would be desirable to obtain a message from the selected VNIC as rapidly as possible.
Traditionally, doorbell interfaces have been used to signal the need to service a separate data queue. However the signaling function associated with doorbell interfaces was separate from the data queue and, therefore, could not be used to also assist in expediting data transfer. Accordingly, it would be desirable to provide methods and systems for communicating data which overcome these drawbacks and limitations.